Pneumonia Detector from X-ray Images
Background:
Chest radiography (X-ray) remains the most common radiological exam in the world and done as a first screening tool for a wide range of conditions (Raoof et al., 2012). In 2006, it is estimated that 129 million CXR images were acquired in the United States alone. The demand comes from low-cost and low radiation dose, and reasonable sensitivity to a wide variety of pathologies.
The interpretation of the chest radiograph can be challenging due to the superimposition of anatomical structures along the projection direction. This effect can make it very difficult to detect abnormalities in particular locations. This causes radiologists to show high inter-observer variability in their analysis of the images (Balabanova et al., 2005).
The volume of images acquired, the complexity of their interpretation, and their value in clinical practice have long motivated researchers to build automated algorithms for analysis. The trend in using modern machine learning techniques started in 2017 when NIH released 112,000 images from NIH clinical centers. And in 2019 more than 750,000 additional images were released in 3 labeled datasets.
The potential gains from ML-based automated analysis include increased sensitivity for subtle findings, prioritization of time-sensitive cases, automation of tedious daily tasks, and provision of analysis in situations where radiologists are not available. In addition, such tools can also be a first step before an experienced doctors reviews the result. It would increase confidence and reduce overall false positive or false negative diagnosis.
Problem Statement:
The goal is to reliably predict the probability of Pneumonia in a patient from the chest x-ray. While many papers are published in this field. It seems the focus has been on architecture of CNN that can achieve the classification. While in medical field an accurate estimate of the probability is critical when making diagnosis.
Solution Statement:
We intend to train a CNN model to classify the X-ray images into Normal and Pneumonia. For more efficient and faster training we load weights from a pre-trained ResNet model. A softmax at the output of the last layer gives us the uncalibrated probabilities of the Pneumonia vs. Normal.
We train another model using beta calibration model to go from uncalibrated probabilities into calibrated probabilities. The concept for calibration is the if for example the probability of Pneumonia is $p \%$ for some patients, we expect that actually $p \%$ of such patients have Pneumonia. This is a much more useful information to the physician that need to make decisions for the patient.
To calibrate this second model, we first need to split our training dataset into train and calibration sets. The calibration set does not need to be large as there are not so many parameters to tune.
After the calibration model is trained/fitted. The outputs of the CNN is softmaxed and passed to the calibration model to determine the probability of the Pneumonia from the patient's X-ray.
Datasets:
There are several chest X-ray datasets available from different sources. We downloaded this Dataset from Kaggle. The Original source of data is University of California San Diego in here
The dataset consist of 5840 total 2D Black and White X-ray images. The dimension of the images varies in the dataset and therefore a consistent resizing is needed. The dimensions are in the range of (900 to 1400) x (600 to 1300). When doing resizing we should take note of possible image aliasing and apply necessary smoothing filters if needed.
The labels are binary as NORMAL and PNEUMONIA. The train dataset has 3875/5216 images of PNEUMONIA and 1341/5216 of NORMAL. The test dataset 390/624 PNEUMONIA and 234/624 NORMAL images.
From the train dataset we pick 100 random images from Normal and 100 random images from Pneumonia and use them only for training the calibration model.
Benchmark Model:
For accuracy, precision, recall the model would be compared to: Kermany et. al.30154-5)
The benchmark for probability calibration is the dataset itself through a reliability diagram on the test dataset.
Evaluation Metrics:
a. Accuracy, Precision, Recall, F1-score :
The overall accuracy of model should be high for the probability calibration to work properly. In this application Precision and Recall are both important as both False Positives and False Negative can impact the medical procedures that would be applied or not applied to a patient. Therefore, a combined F1-score can also be a suitable metric.
b. Reliability Diagram :
We will look at probability diagram over the entire dataset. Make predictions based on calibrated probabilities and discuss the results.
Install and import on the instance
In [6]:
%%capture
import sys
!{sys.executable} -m pip install smdebug torch torchvision tqdm opendatasets betacal ml-insights==0.1.8
In [3]:
import ml_insights
ml_insights.__version__
Out[3]:
'0.1.8'
In [56]:
import os
import boto3
import sagemaker
from sagemaker.tuner import CategoricalParameter, ContinuousParameter, HyperparameterTuner
from sagemaker.pytorch import PyTorch
from sagemaker import get_execution_role
from sagemaker.debugger import Rule, DebuggerHookConfig, TensorBoardOutputConfig, CollectionConfig, ProfilerRule, rule_configs, ProfilerConfig, FrameworkProfile
from sagemaker.analytics import HyperparameterTuningJobAnalytics
import opendatasets as op
import numpy as np
import random
Get the data and copy it to S3
In [ ]:
%%capture
!pip install -U sagemaker
Download the data into notebook instance
In [6]:
op.download('https://www.kaggle.com/datasets/tolgadincer/labeled-chest-xray-images')
Downloading labeled-chest-xray-images.zip to ./labeled-chest-xray-images
100%|██████████| 1.17G/1.17G [00:10<00:00, 120MB/s]
Copy the data into S3 bucket...
In [5]:
session = sagemaker.Session()
bucket = session.default_bucket()
region = session.boto_region_name
role = sagemaker.get_execution_role()
os.environ["DEFAULT_S3_BUCKET"] = bucket
In [9]:
%%capture
!aws s3 sync ./labeled-chest-xray-images s3://${DEFAULT_S3_BUCKET}/chestCRXImages/
In [6]:
print(bucket)
s3bucket = os.path.join("s3://", bucket)
print(s3bucket)
s3datapath = os.path.join(s3bucket, "chestCRXImages/chest_xray")
print(s3datapath)
sagemaker-us-east-1-286375333242 s3://sagemaker-us-east-1-286375333242 s3://sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray
Make a list of all the NORMAL and PNEUMONIA images in the train dataset. This is used to split the train dataset randomly into train and calibration...
In [7]:
s3 = boto3.resource('s3')
my_bucket = s3.Bucket(bucket)
NORMAL_images = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/train/NORMAL"):
#print(object_summary.key)
NORMAL_images.append(object_summary.key)
PNEUMONIA_images = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/train/PNEUMONIA"):
#print(object_summary.key)
PNEUMONIA_images.append(object_summary.key)
In [8]:
print(len(NORMAL_images))
print(len(PNEUMONIA_images))
1349 3883
In [12]:
Normal_len = len(NORMAL_images)
Pneumonia_len = len(PNEUMONIA_images)
Normal_index = random.sample(range(Normal_len), 100)
Pneumonia_index = random.sample(range(Pneumonia_len), 100)
Normal_images_calibration = [NORMAL_images[index] for index in Normal_index]
Pneumonia_images_calibration = [PNEUMONIA_images[index] for index in Pneumonia_index]
Calibration_images = Normal_images_calibration + Pneumonia_images_calibration
print("Moving some random files into Calibration folder...")
print("Bucker is " + bucket)
target_str = 'train/'
for image_name in Calibration_images:
s_pos = image_name.index(target_str)
new_image_name = image_name[:s_pos] + "Calibration/" + image_name[s_pos + len(target_str):]
source_loc = bucket + '/' + image_name
print('----------------------------')
print("Moving " + source_loc + " to " + new_image_name)
s3.Object(bucket, new_image_name).copy_from(CopySource=source_loc);
print("Deleting " + source_loc)
s3.Object(bucket, source_loc).delete();
Moving some random files into Calibration folder... Bucker is sagemaker-us-east-1-286375333242 ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-824917-0003.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-824917-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-824917-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7298141-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7298141-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7298141-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4969331-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4969331-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4969331-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8181293-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8181293-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8181293-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8272933-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8272933-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8272933-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4507677-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4507677-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4507677-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4190635-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4190635-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4190635-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3852346-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3852346-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3852346-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8412621-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8412621-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8412621-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6973172-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6973172-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6973172-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3747940-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3747940-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3747940-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4271375-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4271375-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-4271375-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9742274-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9742274-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9742274-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8644314-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8644314-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8644314-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9084382-0003.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9084382-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9084382-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6081265-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6081265-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6081265-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8229007-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8229007-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8229007-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1522956-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1522956-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1522956-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5708209-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5708209-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5708209-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5918516-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5918516-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5918516-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6901859-0004.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6901859-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6901859-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2333546-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2333546-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2333546-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7103127-0003.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7103127-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7103127-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-263932-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-263932-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-263932-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8799208-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8799208-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8799208-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3997421-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3997421-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3997421-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6044251-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6044251-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6044251-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8688367-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8688367-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8688367-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8903542-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8903542-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8903542-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9697349-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9697349-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9697349-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-202916-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-202916-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-202916-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7696091-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7696091-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7696091-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-974509-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-974509-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-974509-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5176049-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5176049-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5176049-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1044645-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1044645-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1044645-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3175613-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3175613-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3175613-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2048417-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2048417-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2048417-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5759621-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5759621-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-5759621-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2387301-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2387301-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2387301-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1830070-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1830070-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1830070-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7001302-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7001302-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7001302-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1128157-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1128157-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-1128157-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2239132-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2239132-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-2239132-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9033279-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9033279-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9033279-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3487615-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3487615-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-3487615-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7436212-0003.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7436212-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-7436212-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8439083-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8439083-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8439083-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9430515-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9430515-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-9430515-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6915237-0001.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6915237-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-6915237-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8998369-0002.jpeg to chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8998369-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/NORMAL/NORMAL-8998369-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5150733-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5150733-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5150733-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5503803-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5503803-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5503803-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5882850-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5882850-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5882850-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9566232-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9566232-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9566232-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5724853-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5724853-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-5724853-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-324273-0008.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-324273-0008.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-324273-0008.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6065052-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6065052-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6065052-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8313249-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8313249-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8313249-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9269035-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9269035-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9269035-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8034950-0012.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8034950-0012.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8034950-0012.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9487238-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9487238-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9487238-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-1612812-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1612812-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-1612812-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1191925-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1191925-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1191925-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-4057077-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4057077-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-4057077-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-3371316-0005.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-3371316-0005.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-3371316-0005.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6839027-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6839027-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6839027-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1335423-0010.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1335423-0010.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1335423-0010.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6335562-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6335562-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6335562-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4815049-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4815049-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4815049-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4320648-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4320648-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4320648-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-7432256-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7432256-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-7432256-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-463487-0007.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-463487-0007.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-463487-0007.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-4401421-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4401421-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-4401421-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6823523-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6823523-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6823523-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5701772-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5701772-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5701772-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-3722564-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-3722564-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-3722564-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7299583-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7299583-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7299583-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-2814755-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-2814755-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-2814755-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-463487-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-463487-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-463487-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7564600-0007.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7564600-0007.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7564600-0007.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8241352-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8241352-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8241352-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8797622-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8797622-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-8797622-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-1546263-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1546263-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-1546263-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-2180985-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-2180985-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-2180985-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5615173-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5615173-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5615173-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-3000214-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-3000214-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-3000214-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-741892-0003.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-741892-0003.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-741892-0003.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1801584-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1801584-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1801584-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4016460-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4016460-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4016460-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5221323-0004.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5221323-0004.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5221323-0004.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6669017-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6669017-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-6669017-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-2092977-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-2092977-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-2092977-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5285130-0006.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5285130-0006.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-5285130-0006.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7792467-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7792467-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-7792467-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4671230-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4671230-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-4671230-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1245228-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1245228-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-1245228-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-966858-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-966858-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-966858-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-428676-0002.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-428676-0002.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/VIRUS-428676-0002.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9332973-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9332973-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-9332973-0001.jpeg ---------------------------- Moving sagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-649767-0001.jpeg to chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-649767-0001.jpeg Deletingsagemaker-us-east-1-286375333242/chestCRXImages/chest_xray/train/PNEUMONIA/BACTERIA-649767-0001.jpeg
Now the data is ready on S3 and is plit into train, calibration, and test and ready for training.
Data Exploration
To keep the notebook simplier and easier to follow the data exploration is skipped. X-ray images would be shown later in the notebook when inference is done.
Early on exploration of the X-ray images showed that two main pre-processing need to happen:
- Resizing the images because the size of input images can vary
- Since there are BW and our pre-trained model is for 3-color channel images, we need to convert the images from BW to RGB. When using a dataloader from pytorch during the script training, this is done automatically - But when we want to send the image data to the endpoint for inference we need to explicity to this conversion as we will see later in the notebook.
Data Exploration
First, we do a hyper-parameter tuning of the model.
The training script is in CNN_training.py It is based on ResNet50 pre-trained model. The last dense/Fully-connected layer is replaced by two layers. One with 128 nodes and the last layer is with two nodes for the binary classification. The code is uploaded to the github folder.
For hyper-parameter tuning we can search for optimum values of learning rate and batch size, while aiming to minimize the testing loss.
Hyper-parameter Tuning
In [13]:
hyperparameter_ranges = {
"learning_rate": ContinuousParameter(0.001, 0.01),
"batch_size": CategoricalParameter([16, 32, 64, 128]),
}
role = sagemaker.get_execution_role()
objective_metric_name = "Test Loss"
objective_type = "Minimize"
metric_definitions = [{"Name": "Test Loss", "Regex": "Testing Loss: ([0-9\\.]+)"}]
In [14]:
estimator = PyTorch(
entry_point="CNN_training.py",
base_job_name='CRX_Pneumonia',
role=role,
framework_version="1.8",
instance_count=1,
instance_type="ml.m5.large",
py_version='py36'
)
tuner = HyperparameterTuner(
estimator,
objective_metric_name,
hyperparameter_ranges,
metric_definitions,
max_jobs=2,
max_parallel_jobs=1, # you once have one ml.g4dn.xlarge instance available
objective_type=objective_type
)
Fit the tuner
In [14]:
os.environ['SM_CHANNEL_TRAINING']=s3datapath
os.environ['SM_MODEL_DIR']=os.path.join(s3datapath, "model")#'s3://udacitysolution/model/'
os.environ['SM_OUTPUT_DATA_DIR']=os.path.join(s3datapath, "output")#'s3://udacitysolution/output/'
tuner.fit({"training": s3datapath})
No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config
...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................!
In [15]:
#or find under sagemaker hyper-parameter tuning jobs if kernel dies
#latest_tuning_job_name = tuner.latest_tuning_job.name
latest_tuning_job_name = "pytorch-training-220921-0404"
print(latest_tuning_job_name)
pytorch-training-220921-0404
Tuning results
In [16]:
exp = HyperparameterTuningJobAnalytics(
hyperparameter_tuning_job_name=latest_tuning_job_name)
jobs = exp.dataframe()
jobs.sort_values('FinalObjectiveValue', ascending=0)
Out[16]:
| batch_size | learning_rate | TrainingJobName | TrainingJobStatus | FinalObjectiveValue | TrainingStartTime | TrainingEndTime | TrainingElapsedTimeSeconds | |
|---|---|---|---|---|---|---|---|---|
| 1 | "32" | 0.003989 | pytorch-training-220921-0404-001-55d34e19 | Completed | 14.0 | 2022-09-21 04:06:47+00:00 | 2022-09-21 04:46:59+00:00 | 2412.0 |
| 0 | "32" | 0.006130 | pytorch-training-220921-0404-002-551b9194 | Completed | 12.0 | 2022-09-21 04:54:43+00:00 | 2022-09-21 05:34:03+00:00 | 2360.0 |
Imp: If kernel dies, load from sagemaker¶
In [17]:
BetterTrainingJobName='pytorch-training-220921-0404-002-551b9194'
my_estimator = sagemaker.estimator.Estimator.attach(BetterTrainingJobName)
my_estimator.hyperparameters()
best_estimator=my_estimator
2022-09-21 05:34:06 Starting - Found matching resource for reuse 2022-09-21 05:34:06 Downloading - Downloading input data 2022-09-21 05:34:06 Training - Training image download completed. Training in progress. 2022-09-21 05:34:06 Uploading - Uploading generated training model 2022-09-21 05:34:06 Completed - Resource released due to keep alive period expiry
Model training with the best hyper-parameters
In [ ]:
best_estimator=tuner.best_estimator()
In [18]:
best_estimator.hyperparameters()
Out[18]:
{'_tuning_objective_metric': 'Test Loss',
'batch_size': '"32"',
'learning_rate': '0.0061302077895518114',
'sagemaker_container_log_level': '20',
'sagemaker_estimator_class_name': '"PyTorch"',
'sagemaker_estimator_module': '"sagemaker.pytorch.estimator"',
'sagemaker_job_name': '"CRX_Pneumonia-2022-09-21-04-04-41-276"',
'sagemaker_program': '"CNN_training.py"',
'sagemaker_region': '"us-east-1"',
'sagemaker_submit_directory': '"s3://sagemaker-us-east-1-286375333242/CRX_Pneumonia-2022-09-21-04-04-41-276/source/sourcedir.tar.gz"'}
In [19]:
hyperparameters = {"batch_size": int(best_estimator.hyperparameters()['batch_size'].replace('"', '')), \
"lr": best_estimator.hyperparameters()['learning_rate'],
"epochs" : 5}
hyperparameters
Out[19]:
{'batch_size': 32, 'lr': '0.0061302077895518114', 'epochs': 5}
Setting up debugger and profiler for the training job
In [20]:
rules = [
Rule.sagemaker(rule_configs.vanishing_gradient()),
Rule.sagemaker(rule_configs.overfit()),
Rule.sagemaker(rule_configs.overtraining()),
Rule.sagemaker(rule_configs.poor_weight_initialization()),
ProfilerRule.sagemaker(rule_configs.ProfilerReport()),
]
In [21]:
hook_config = DebuggerHookConfig(
hook_parameters={
"train.save_interval": "1",
"eval.save_interval": "1"
}
)
profiler_config = ProfilerConfig(
system_monitor_interval_millis=500, framework_profile_params=FrameworkProfile(num_steps=1)
)
Creating an Estimator
In [22]:
estimator = PyTorch(
entry_point='CNN_training.py',
base_job_name='CXR-Pneumonia',
role=role,
instance_count=1,
instance_type='ml.p3.2xlarge',
framework_version='1.8',
py_version='py36',
hyperparameters=hyperparameters,
## Debugger and Profiler parameters
rules = rules,
debugger_hook_config=hook_config,
profiler_config=profiler_config,
)
Training the model
In [23]:
estimator.fit({"training": s3datapath}, wait = True)
2022-09-28 07:17:57 Starting - Starting the training job... 2022-09-28 07:18:23 Starting - Preparing the instances for trainingVanishingGradient: InProgress Overfit: InProgress Overtraining: InProgress PoorWeightInitialization: InProgress ProfilerReport: InProgress ......... 2022-09-28 07:19:57 Downloading - Downloading input data......... 2022-09-28 07:21:18 Training - Downloading the training image..................... 2022-09-28 07:24:59 Training - Training image download completed. Training in progress..bash: cannot set terminal process group (-1): Inappropriate ioctl for device bash: no job control in this shell 2022-09-28 07:24:56,447 sagemaker-training-toolkit INFO Imported framework sagemaker_pytorch_container.training 2022-09-28 07:24:56,479 sagemaker_pytorch_container.training INFO Block until all host DNS lookups succeed. 2022-09-28 07:24:56,486 sagemaker_pytorch_container.training INFO Invoking user training script. 2022-09-28 07:24:57,036 sagemaker-training-toolkit INFO Invoking user script Training Env: { "additional_framework_parameters": {}, "channel_input_dirs": { "training": "/opt/ml/input/data/training" }, "current_host": "algo-1", "framework_module": "sagemaker_pytorch_container.training:main", "hosts": [ "algo-1" ], "hyperparameters": { "batch_size": 32, "epochs": 5, "lr": "0.0061302077895518114" }, "input_config_dir": "/opt/ml/input/config", "input_data_config": { "training": { "TrainingInputMode": "File", "S3DistributionType": "FullyReplicated", "RecordWrapperType": "None" } }, "input_dir": "/opt/ml/input", "is_master": true, "job_name": "CXR-Pneumonia-2022-09-28-07-17-57-174", "log_level": 20, "master_hostname": "algo-1", "model_dir": "/opt/ml/model", "module_dir": "s3://sagemaker-us-east-1-286375333242/CXR-Pneumonia-2022-09-28-07-17-57-174/source/sourcedir.tar.gz", "module_name": "CNN_training", "network_interface_name": "eth0", "num_cpus": 8, "num_gpus": 1, "output_data_dir": "/opt/ml/output/data", "output_dir": "/opt/ml/output", "output_intermediate_dir": "/opt/ml/output/intermediate", "resource_config": { "current_host": "algo-1", "current_instance_type": "ml.p3.2xlarge", "current_group_name": "homogeneousCluster", "hosts": [ "algo-1" ], "instance_groups": [ { "instance_group_name": "homogeneousCluster", "instance_type": "ml.p3.2xlarge", "hosts": [ "algo-1" ] } ], "network_interface_name": "eth0" }, "user_entry_point": "CNN_training.py" } Environment variables: SM_HOSTS=["algo-1"] SM_NETWORK_INTERFACE_NAME=eth0 SM_HPS={"batch_size":32,"epochs":5,"lr":"0.0061302077895518114"} SM_USER_ENTRY_POINT=CNN_training.py SM_FRAMEWORK_PARAMS={} SM_RESOURCE_CONFIG={"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.p3.2xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.p3.2xlarge"}],"network_interface_name":"eth0"} SM_INPUT_DATA_CONFIG={"training":{"RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}} SM_OUTPUT_DATA_DIR=/opt/ml/output/data SM_CHANNELS=["training"] SM_CURRENT_HOST=algo-1 SM_MODULE_NAME=CNN_training SM_LOG_LEVEL=20 SM_FRAMEWORK_MODULE=sagemaker_pytorch_container.training:main SM_INPUT_DIR=/opt/ml/input SM_INPUT_CONFIG_DIR=/opt/ml/input/config SM_OUTPUT_DIR=/opt/ml/output SM_NUM_CPUS=8 SM_NUM_GPUS=1 SM_MODEL_DIR=/opt/ml/model SM_MODULE_DIR=s3://sagemaker-us-east-1-286375333242/CXR-Pneumonia-2022-09-28-07-17-57-174/source/sourcedir.tar.gz SM_TRAINING_ENV={"additional_framework_parameters":{},"channel_input_dirs":{"training":"/opt/ml/input/data/training"},"current_host":"algo-1","framework_module":"sagemaker_pytorch_container.training:main","hosts":["algo-1"],"hyperparameters":{"batch_size":32,"epochs":5,"lr":"0.0061302077895518114"},"input_config_dir":"/opt/ml/input/config","input_data_config":{"training":{"RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}},"input_dir":"/opt/ml/input","is_master":true,"job_name":"CXR-Pneumonia-2022-09-28-07-17-57-174","log_level":20,"master_hostname":"algo-1","model_dir":"/opt/ml/model","module_dir":"s3://sagemaker-us-east-1-286375333242/CXR-Pneumonia-2022-09-28-07-17-57-174/source/sourcedir.tar.gz","module_name":"CNN_training","network_interface_name":"eth0","num_cpus":8,"num_gpus":1,"output_data_dir":"/opt/ml/output/data","output_dir":"/opt/ml/output","output_intermediate_dir":"/opt/ml/output/intermediate","resource_config":{"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.p3.2xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.p3.2xlarge"}],"network_interface_name":"eth0"},"user_entry_point":"CNN_training.py"} SM_USER_ARGS=["--batch_size","32","--epochs","5","--lr","0.0061302077895518114"] SM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate SM_CHANNEL_TRAINING=/opt/ml/input/data/training SM_HP_BATCH_SIZE=32 SM_HP_EPOCHS=5 SM_HP_LR=0.0061302077895518114 PYTHONPATH=/opt/ml/code:/opt/conda/bin:/opt/conda/lib/python36.zip:/opt/conda/lib/python3.6:/opt/conda/lib/python3.6/lib-dynload:/opt/conda/lib/python3.6/site-packages Invoking script with the following command: /opt/conda/bin/python3.6 CNN_training.py --batch_size 32 --epochs 5 --lr 0.0061302077895518114 [2022-09-28 07:25:00.116 algo-1:26 INFO utils.py:27] RULE_JOB_STOP_SIGNAL_FILENAME: None [2022-09-28 07:25:00.235 algo-1:26 INFO profiler_config_parser.py:102] Using config at /opt/ml/input/config/profilerconfig.json. Namespace(batch_size=32, data='/opt/ml/input/data/training', epochs=5, lr=0.0061302077895518114, model_dir='/opt/ml/model', output_dir='/opt/ml/output/data', test_dir='/opt/ml/input/data/training/test', train_dir='/opt/ml/input/data/training/train') Hyperparameters are LR: 0.0061302077895518114, Batch Size: 32 Epochs: 5 Data Paths: /opt/ml/input/data/training Available device is: cuda:0 [2022-09-28 07:25:07.642 algo-1:26 INFO json_config.py:91] Creating hook from json_config at /opt/ml/input/config/debughookconfig.json. [2022-09-28 07:25:07.644 algo-1:26 INFO hook.py:201] tensorboard_dir has not been set for the hook. SMDebug will not be exporting tensorboard summaries. [2022-09-28 07:25:07.646 algo-1:26 INFO hook.py:255] Saving to /opt/ml/output/tensors [2022-09-28 07:25:07.646 algo-1:26 INFO state_store.py:77] The checkpoint config file /opt/ml/input/config/checkpointconfig.json does not exist. [2022-09-28 07:25:07.677 algo-1:26 INFO hook.py:591] name:fc.0.weight count_params:262144 [2022-09-28 07:25:07.678 algo-1:26 INFO hook.py:591] name:fc.0.bias count_params:128 [2022-09-28 07:25:07.678 algo-1:26 INFO hook.py:591] name:fc.2.weight count_params:256 [2022-09-28 07:25:07.679 algo-1:26 INFO hook.py:591] name:fc.2.bias count_params:2 [2022-09-28 07:25:07.679 algo-1:26 INFO hook.py:593] Total Trainable Params: 262530 Starting Model Training Hyperparameters: epoch: 5, lr: 0.0061302077895518114, batch size: 32 [2022-09-28 07:25:08.710 algo-1:26 INFO hook.py:425] Monitoring the collections: gradients, relu_input, losses [2022-09-28 07:25:08.712 algo-1:26 INFO python_profiler.py:182] Dumping cProfile stats to /opt/ml/output/profiler/framework/pytorch/cprofile/26-algo-1/prestepzero-*-start-1664349900235821.5_train-0-stepstart-1664349908711716.2/python_stats. [2022-09-28 07:25:08.727 algo-1:26 INFO hook.py:488] Hook is writing from the hook with pid: 26 [2022-09-28 07:25:21.492 algo-1:26 INFO python_profiler.py:182] Dumping cProfile stats to /opt/ml/output/profiler/framework/pytorch/cprofile/26-algo-1/train-0-stepstart-1664349908723235.8_train-0-forwardpassend-1664349921492455.8/python_stats. [2022-09-28 07:25:22.161 algo-1:26 INFO python_profiler.py:182] Dumping cProfile stats to /opt/ml/output/profiler/framework/pytorch/cprofile/26-algo-1/train-0-forwardpassend-1664349921495912.8_train-1-stepstart-1664349922160183.8/python_stats. Train epoch 1: [320/5232 (6%)]#011Loss: 0.375405 Train epoch 1: [640/5232 (12%)]#011Loss: 0.206932 Train epoch 1: [960/5232 (18%)]#011Loss: 0.438136 Train epoch 1: [1280/5232 (24%)]#011Loss: 0.439663 Train epoch 1: [1600/5232 (30%)]#011Loss: 0.494468 Train epoch 1: [1920/5232 (37%)]#011Loss: 0.133440 Train epoch 1: [2240/5232 (43%)]#011Loss: 0.387662 Train epoch 1: [2560/5232 (49%)]#011Loss: 0.207986 Train epoch 1: [2880/5232 (55%)]#011Loss: 0.222797 Train epoch 1: [3200/5232 (61%)]#011Loss: 0.223536 Train epoch 1: [3520/5232 (67%)]#011Loss: 0.493388 Train epoch 1: [3840/5232 (73%)]#011Loss: 0.152841 Train epoch 1: [4160/5232 (79%)]#011Loss: 0.514780 Train epoch 1: [4480/5232 (85%)]#011Loss: 0.292197 Train epoch 1: [4800/5232 (91%)]#011Loss: 0.285628 Train epoch 1: [5120/5232 (98%)]#011Loss: 0.224347 Testing... Test set: Average loss: 0.0134, Test Accuracy: 84.61538461538461% Train epoch 2: [320/5232 (6%)]#011Loss: 0.143077 Train epoch 2: [640/5232 (12%)]#011Loss: 0.202894 Train epoch 2: [960/5232 (18%)]#011Loss: 0.096761 Train epoch 2: [1280/5232 (24%)]#011Loss: 0.292739 Train epoch 2: [1600/5232 (30%)]#011Loss: 0.098056 Train epoch 2: [1920/5232 (37%)]#011Loss: 0.131096 Train epoch 2: [2240/5232 (43%)]#011Loss: 0.163605 Train epoch 2: [2560/5232 (49%)]#011Loss: 0.290103 Train epoch 2: [2880/5232 (55%)]#011Loss: 0.465909 Train epoch 2: [3200/5232 (61%)]#011Loss: 0.327568 Train epoch 2: [3520/5232 (67%)]#011Loss: 0.215384 Train epoch 2: [3840/5232 (73%)]#011Loss: 0.339750 Train epoch 2: [4160/5232 (79%)]#011Loss: 0.301169 Train epoch 2: [4480/5232 (85%)]#011Loss: 0.177459 Train epoch 2: [4800/5232 (91%)]#011Loss: 0.140137 Train epoch 2: [5120/5232 (98%)]#011Loss: 0.334776 Testing... Test set: Average loss: 0.0105, Test Accuracy: 85.25641025641026% Train epoch 3: [320/5232 (6%)]#011Loss: 0.320791 Train epoch 3: [640/5232 (12%)]#011Loss: 0.113965 Train epoch 3: [960/5232 (18%)]#011Loss: 0.328466 Train epoch 3: [1280/5232 (24%)]#011Loss: 0.139991 Train epoch 3: [1600/5232 (30%)]#011Loss: 0.213227 Train epoch 3: [1920/5232 (37%)]#011Loss: 0.265513 Train epoch 3: [2240/5232 (43%)]#011Loss: 0.318187 Train epoch 3: [2560/5232 (49%)]#011Loss: 0.189421 Train epoch 3: [2880/5232 (55%)]#011Loss: 0.247238 Train epoch 3: [3200/5232 (61%)]#011Loss: 0.358383 Train epoch 3: [3520/5232 (67%)]#011Loss: 0.305215 Train epoch 3: [3840/5232 (73%)]#011Loss: 0.225445 Train epoch 3: [4160/5232 (79%)]#011Loss: 0.175306 Train epoch 3: [4480/5232 (85%)]#011Loss: 0.355496 Train epoch 3: [4800/5232 (91%)]#011Loss: 0.203509 Train epoch 3: [5120/5232 (98%)]#011Loss: 0.329521 Testing... Test set: Average loss: 0.0107, Test Accuracy: 84.61538461538461% Train epoch 4: [320/5232 (6%)]#011Loss: 0.404709 VanishingGradient: InProgress Overfit: IssuesFound Overtraining: InProgress PoorWeightInitialization: InProgress Train epoch 4: [640/5232 (12%)]#011Loss: 0.227930 Train epoch 4: [960/5232 (18%)]#011Loss: 0.142792 Train epoch 4: [1280/5232 (24%)]#011Loss: 0.253626 Train epoch 4: [1600/5232 (30%)]#011Loss: 0.301804 Train epoch 4: [1920/5232 (37%)]#011Loss: 0.212710 Train epoch 4: [2240/5232 (43%)]#011Loss: 0.214941 Train epoch 4: [2560/5232 (49%)]#011Loss: 0.316854 Train epoch 4: [2880/5232 (55%)]#011Loss: 0.128724 Train epoch 4: [3200/5232 (61%)]#011Loss: 0.253114 Train epoch 4: [3520/5232 (67%)]#011Loss: 0.312859 Train epoch 4: [3840/5232 (73%)]#011Loss: 0.294126 Train epoch 4: [4160/5232 (79%)]#011Loss: 0.204234 Train epoch 4: [4480/5232 (85%)]#011Loss: 0.131046 Train epoch 4: [4800/5232 (91%)]#011Loss: 0.227132 Train epoch 4: [5120/5232 (98%)]#011Loss: 0.141236 Testing... VanishingGradient: InProgress Overfit: IssuesFound Overtraining: IssuesFound PoorWeightInitialization: InProgress Test set: Average loss: 0.0193, Test Accuracy: 77.40384615384616% Train epoch 5: [320/5232 (6%)]#011Loss: 0.346485 Train epoch 5: [640/5232 (12%)]#011Loss: 0.147757 Train epoch 5: [960/5232 (18%)]#011Loss: 0.200689 Train epoch 5: [1280/5232 (24%)]#011Loss: 0.216724 Train epoch 5: [1600/5232 (30%)]#011Loss: 0.276936 Train epoch 5: [1920/5232 (37%)]#011Loss: 0.228897 Train epoch 5: [2240/5232 (43%)]#011Loss: 0.488071 Train epoch 5: [2560/5232 (49%)]#011Loss: 0.228150 Train epoch 5: [2880/5232 (55%)]#011Loss: 0.442228 Train epoch 5: [3200/5232 (61%)]#011Loss: 0.426490 Train epoch 5: [3520/5232 (67%)]#011Loss: 0.103515 Train epoch 5: [3840/5232 (73%)]#011Loss: 0.224388 Train epoch 5: [4160/5232 (79%)]#011Loss: 0.201727 Train epoch 5: [4480/5232 (85%)]#011Loss: 0.154246 Train epoch 5: [4800/5232 (91%)]#011Loss: 0.397623 Train epoch 5: [5120/5232 (98%)]#011Loss: 0.293890 Testing... 2022-09-28 07:35:16 Uploading - Uploading generated training modelTest set: Average loss: 0.0131, Test Accuracy: 85.41666666666667% Saving the model to /opt/ml/model/model.pth Training Finished... INFO:__main__:Hyperparameters are LR: 0.0061302077895518114, Batch Size: 32 INFO:__main__:Epochs: 5 INFO:__main__:Data Paths: /opt/ml/input/data/training Downloading: "https://download.pytorch.org/models/resnet50-19c8e357.pth" to /root/.cache/torch/hub/checkpoints/resnet50-19c8e357.pth #015 0%| | 0.00/97.8M [00:00<?, ?B/s]#015 5%|▌ | 5.18M/97.8M [00:00<00:01, 54.3MB/s]#015 13%|█▎ | 12.2M/97.8M [00:00<00:01, 66.0MB/s]#015 20%|█▉ | 19.3M/97.8M [00:00<00:01, 69.7MB/s]#015 27%|██▋ | 26.4M/97.8M [00:00<00:01, 71.3MB/s]#015 34%|███▍ | 33.2M/97.8M [00:00<00:00, 71.5MB/s]#015 41%|████▏ | 40.4M/97.8M [00:00<00:00, 72.6MB/s]#015 48%|████▊ | 47.4M/97.8M [00:00<00:00, 72.9MB/s]#015 56%|█████▌ | 54.5M/97.8M [00:00<00:00, 73.3MB/s]#015 63%|██████▎ | 61.5M/97.8M [00:00<00:00, 72.5MB/s]#015 70%|███████ | 68.6M/97.8M [00:01<00:00, 73.1MB/s]#015 77%|███████▋ | 75.7M/97.8M [00:01<00:00, 73.5MB/s]#015 85%|████████▍ | 82.9M/97.8M [00:01<00:00, 74.1MB/s]#015 92%|█████████▏| 90.0M/97.8M [00:01<00:00, 74.4MB/s]#015 99%|█████████▉| 97.1M/97.8M [00:01<00:00, 73.9MB/s]#015100%|██████████| 97.8M/97.8M [00:01<00:00, 72.3MB/s] INFO:__main__:Starting Model Training INFO:__main__:Testing... INFO:__main__: Test set: Average loss: 0.0134, Test Accuracy: 84.61538461538461% INFO:__main__:Testing... INFO:__main__: Test set: Average loss: 0.0105, Test Accuracy: 85.25641025641026% INFO:__main__:Testing... INFO:__main__: Test set: Average loss: 0.0107, Test Accuracy: 84.61538461538461% INFO:__main__:Testing... INFO:__main__: Test set: Average loss: 0.0193, Test Accuracy: 77.40384615384616% INFO:__main__:Testing... INFO:__main__: Test set: Average loss: 0.0131, Test Accuracy: 85.41666666666667% INFO:__main__:Saving the model to /opt/ml/model/model.pth 2022-09-28 07:35:06,554 sagemaker-training-toolkit INFO Reporting training SUCCESS 2022-09-28 07:35:42 Completed - Training job completed ProfilerReport: IssuesFound Training seconds: 955 Billable seconds: 955
Training results
We get around 85% accuracy compared to 90% from the benchmark paper. We could perhaps make the model deeper or increase the training loss to improve the result. It could also be that since we dedicated almost 10% of our Pneumonia images for probability calibration, we did not have enough positive samples for the model to learn.
However, we could also be satisfied with the result. Although, the benchmark reports higher accuracy it does not necessarily be a good model for predicting the probability of Pneumonia in a patient.
Deploy the model
We deploy the model to an endpoint to use it for inference and calibration:
In [24]:
model_location=estimator.model_data
print("Mode data in case of reload or if the kernel dies before deployment:")
print(model_location)
Mode data in case of reload or if the kernel dies before deployment: s3://sagemaker-us-east-1-286375333242/CXR-Pneumonia-2022-09-28-07-17-57-174/output/model.tar.gz
In [17]:
import sagemaker
import boto3
from sagemaker.tuner import CategoricalParameter, ContinuousParameter, HyperparameterTuner
from sagemaker.pytorch import PyTorch
from sagemaker import get_execution_role
from sagemaker.debugger import Rule, DebuggerHookConfig, TensorBoardOutputConfig, CollectionConfig, ProfilerRule, rule_configs
from sagemaker.debugger import ProfilerConfig, FrameworkProfile
from sagemaker.pytorch import PyTorchModel
from sagemaker.predictor import Predictor
In [18]:
jpeg_serializer = sagemaker.serializers.IdentitySerializer("image/jpeg")
json_deserializer = sagemaker.deserializers.JSONDeserializer()
class ImagePredictor(Predictor):
def __init__(self, endpoint_name, sagemaker_session):
super(ImagePredictor, self).__init__(
endpoint_name,
sagemaker_session=sagemaker_session,
serializer=jpeg_serializer,
deserializer=json_deserializer,
)
In [27]:
pytorch_model = PyTorchModel(model_data=model_location, role=role, entry_point='inference.py',py_version='py36',
framework_version='1.8',
predictor_cls=ImagePredictor)
In [28]:
predictor = pytorch_model.deploy(initial_instance_count=1, instance_type='ml.m5.large')
------!
In [14]:
#endpoint_name = predictor.endpoint_name
endpoint_name = "pytorch-inference-2022-09-28-07-40-07-421"
print("The endpoint name is:")
print(endpoint_name)
The endpoint name is: pytorch-inference-2022-09-28-07-40-07-421
Run this cell if the kernel dies but the endpoint is still active:
In [19]:
predictor = sagemaker.predictor.Predictor(
endpoint_name = endpoint_name,
sagemaker_session = session,
serializer = jpeg_serializer,
deserializer = json_deserializer
)
In [ ]:
Probability Calibration
Now, we start on training the calibration model. First, we create a list of all the X-ray images we have available for training the model:
In [9]:
s3 = boto3.resource('s3')
my_bucket = s3.Bucket(bucket)
Normal_images_calibration = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/Calibration/NORMAL"):
#print(object_summary.key)
Normal_images_calibration.append(object_summary.key)
Pneumonia_images_calibration = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/Calibration/PNEUMONIA"):
#print(object_summary.key)
Pneumonia_images_calibration.append(object_summary.key)
print(f"Number of Normal Images available for calibration: {len(Normal_images_calibration)}")
print(f"Number of PNEUMONIA Images available for calibration: {len(Pneumonia_images_calibration)}")
Number of Normal Images available for calibration: 99 Number of PNEUMONIA Images available for calibration: 99
Creating a list of all the images (to be sent to the endpoint for inference) and their labels (from their folder)
In [10]:
cal_data = []
cal_y = []
for image_name in Normal_images_calibration:
image_label = 0
cal_data.append(image_name)
cal_y.append(image_label)
#print('----------------------------')
#print("Image: " + image_name)
#print("Label: " + str(image_label))
for image_name in Pneumonia_images_calibration:
image_label = 1
cal_data.append(image_name)
cal_y.append(image_label)
#print('----------------------------')
#print("Image: " + image_name)
#print("Label: " + str(image_label))
Testing the endpoint and visualizing the results
First step is to send the images in the calibration dataset to the model (endpoint) and calulate the probability of the Pneumonia.
Since the last of the CNN was outputing the logits, we need to take the softmax to get the probabilty.
We store the uncalibrated probability of the entire calibration dataset into a list.
For visualization for the first 3 images the X-ray image is shown. For the rest only the predicted label (NORMAL vs. PNEUMONIA) and the uncalibrated probability of PNEUMONIA is printed.
In [12]:
def softmax(x):
return np.exp(x) / np.sum(np.exp(x), axis=0)
In [20]:
from sagemaker.serializers import IdentitySerializer
from PIL import Image
predictor.serializer = IdentitySerializer("image/jpeg")
jpeg_serializer = IdentitySerializer("image/jpeg")
session = boto3.Session()
s3_client = session.client('s3')
s3 = boto3.resource('s3')
bucket_name = bucket
bucket_ = s3.Bucket(bucket_name)
count = 0
uncalibrated_Pneumonia_probabilities_cal = []
for file, label in zip(cal_data, cal_y):
print("--------------------------")
print("Image: " + file)
print("Label: " + str(label))
object = bucket_.Object(file)
response = object.get()
file_stream = response['Body']
img = Image.open(file_stream).convert('RGB')
if count < 3:
display(img)
count += 1
s3_object = s3_client.get_object(Bucket = bucket_name, Key = file)
body = s3_object['Body']
content = body.read()
response = predictor.predict(content, initial_args={'ContentType':'image/jpeg'})
probs = softmax(response[0])
uncalibrated_Pneumonia_probability = probs[1]
print("Uncalibrated Pneumonia Probability: " + str(uncalibrated_Pneumonia_probability))
uncalibrated_Pneumonia_probabilities_cal.append(uncalibrated_Pneumonia_probability)
prediction = np.argmax(response[0]) + 1
prediction = 'NORMAL' if np.argmax(response[0]) == 0 else 'PNEUMONIA'
print("Prediction is: " + prediction)
-------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1044645-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.08247949068642986 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1101909-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.5358327650468722 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1128157-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.5488907091728171 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1455093-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1992966265915107 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1522956-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1827606288369004 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1602738-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.06274437966081287 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1696866-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.055224822244688776 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1739213-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15349568412461392 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1756042-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.044498276455786676 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1830070-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.40198248212398513 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-1917089-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14148137452643758 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-202916-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09611644921460302 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2048417-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.17898001218638768 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2239132-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.39087523194416035 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2239455-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.04567903905561678 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2285600-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18722167000483841 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-229022-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16722136230845724 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2333546-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.24213955750321503 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-2387301-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11582766561999111 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-263932-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11883243416140077 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3072074-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21413455127489442 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3175613-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9768667963699771 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-32326-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.40992600338892177 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3487615-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18149718792345834 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3573766-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07058099275082463 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3644944-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2045191648019463 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3747940-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.17270908559364215 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3781678-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.04859008207883672 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3789615-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07524698076502394 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3821719-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11818122698375742 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3852346-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3163569940923866 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3964800-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4257504588453206 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-3997421-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.12050903460892232 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4190635-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2769702483758563 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4221981-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.03318439940403991 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4271375-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1487281914933459 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4507677-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10719531134798634 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-452722-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16879908978989075 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4758820-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1391297201624191 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4791400-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.44083880262965225 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-4969331-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08543475931914198 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5031579-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.27764134768713755 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5042453-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15900857036103158 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5176049-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6265080514765197 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5236765-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.19394877993559873 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5243995-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.34279450465192246 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-546828-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1351660948880327 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5602301-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9266349934749432 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5650764-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16890664290005175 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5708209-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6191574639786482 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5728847-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10184637376689493 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5759621-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14386811741520833 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-591136-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.05443430324445407 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-5918516-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09368743095945177 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6044251-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14998640544542843 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6081265-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10345038597057617 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6112892-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.33716113240070167 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6459868-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.28618963889596305 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6830488-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.023836801290251033 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6901859-0004.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14631140026944728 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6915237-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15009253393944647 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6973172-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2179695000024495 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-6978858-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.31012884551926634 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7001302-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07397612136526702 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7019096-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11799577966372278 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7103127-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1040081669573553 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-715046-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16665594220857846 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7298141-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1507403553019537 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7436212-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08336579352185146 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7451061-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.45792367243938403 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7575347-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.17249847573438598 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-7696091-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.056311185851588674 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8033319-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.0487115187933637 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8116905-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10552240784133801 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8143931-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.055859041368373365 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8181293-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.0687771835432738 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8229007-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14495614973038862 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-824917-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14831707139588418 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8272933-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9809221099662029 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8412621-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1708985161044502 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8439083-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.04494916466057478 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8644314-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.22906245390500732 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8654141-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.030356048050112227 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8688367-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10181833078155411 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8799208-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7869254936532897 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8825351-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09508054589455592 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8903542-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.26030302705502567 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-8998369-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.13035805894171412 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9033279-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5830863156152283 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9084382-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1133765398920874 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-933709-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18923549211005983 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9430515-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.05797977427030519 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9571602-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15681203457769088 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9585882-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.20611053541580168 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9697349-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.37376775775751203 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9742274-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.396216510792556 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-974509-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.06368984913850202 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9814816-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1561747034367301 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/NORMAL/NORMAL-9816293-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.057285803587118614 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1086099-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981553935078319 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1150345-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997655531671225 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1170996-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.984739469872657 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1546263-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5336009581878914 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1612812-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9987920150881637 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1620679-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9965266037370154 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-1865585-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992025606191263 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-2092977-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8647898048541489 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-2341120-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.989019291461251 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-2355867-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9989332187267973 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-292199-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998715264589907 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-3000214-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996011094816356 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-3067831-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997341959390899 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-3371316-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993022080147703 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-387616-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.998878604566256 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4016460-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5867859651337504 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4241544-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8732625157959566 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4320648-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.99296979261173 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4519487-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9924192044439597 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4615614-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9881791578825608 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4671230-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9973669470522887 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4766617-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.982326922839557 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-4815049-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994677899826245 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5121478-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9914795677882827 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5150733-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9885862178307646 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-516152-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9094030080212272 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5164118-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9818257699276405 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5285130-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9837420264001984 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5401887-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9691569749885933 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5437440-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992350427264787 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5437440-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999813929883065 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5465856-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999270502942858 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5503803-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.625701318723582 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5678659-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9127759742159963 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5724853-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9862907958626117 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-5839760-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.3198851468210602 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6065052-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9948127926421738 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6335562-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999570180784473 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-649767-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9407494087426158 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6512753-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9728592531821827 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6592427-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5997886365743098 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6610440-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.767732599785662 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6669017-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997597419916731 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6823523-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999591161250703 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-6839027-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9846301273913991 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7035571-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9959121729271968 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-741892-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5007508354205628 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7432256-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9958268396857259 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7676769-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9860810252793689 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7721065-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9365190127905303 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7746316-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992057387873071 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-7882471-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9506276753112615 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8034950-0012.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9092655942122009 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8241352-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996133598925245 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8313249-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9957078026325431 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8498464-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999761711292495 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8545367-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996425048994109 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8660561-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985293157369308 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-8797622-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9922640097350709 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9026424-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9991413401990586 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9269035-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5087190790672349 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9332973-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9899304322032237 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9487238-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9913062823629186 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9566232-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9965060815891819 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-960851-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9963740128096463 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/BACTERIA-9674936-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994489579914886 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1191925-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9375112197644844 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1245228-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.911324861881724 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1335423-0010.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985639623127337 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-1801584-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9352787429084739 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-2110654-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8601197608104668 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-2180985-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.3780564916764594 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-2814755-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9904472507750794 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-324273-0008.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9886667599131377 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-3722564-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999492805626032 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-3829565-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993382617234827 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4057077-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9408004461690002 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4271267-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9035972745746248 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-428676-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.993536614259515 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4401421-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9924479588208645 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-4617189-0009.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9907120525642635 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-463487-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9931707488230604 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-463487-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981845492718253 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5066255-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981793871548416 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5150733-0015.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8872403178817235 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5221323-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9953766784821969 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5285130-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993319796974999 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5331068-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9955521186663538 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5615173-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993523885328185 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5701772-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9988943978172292 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-5882850-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8707910850304751 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-6718152-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9388836772150617 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7299583-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993855628937748 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7564600-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6386828012668863 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7792467-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985502199027168 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-7986969-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9812640620318392 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-8268782-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9893356151283801 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-8797622-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.4864939085443749 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/Calibration/PNEUMONIA/VIRUS-966858-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9742854952603427 Prediction is: PNEUMONIA
The calibration dataset does not have many samples and its visualization may not show enough information. However, we can still get an idea before looking into the entire test dataset and before calibration the probabilities.
We can see the most of the images result in either high probability or low probability, however these regions are important for calibration purposes and cannot be ignored as most of the physician errors coudl come from these areas. For other sections with not much points the data could be noisy.
In general from the figure we can learn that our current model over-estimated the probability of Pneumonia from the X-ray images as the predicted probabilties are generally higher than actual probabilties.
In [22]:
import matplotlib.pyplot as plt
from sklearn.calibration import calibration_curve, CalibrationDisplay
plt.figure(figsize=(10,5))
n, bins, patches = plt.hist(x=uncalibrated_Pneumonia_probabilities_cal, bins=5, color='#0504aa', alpha=0.5, rwidth=0.75)
plt.figure(figsize=(10,5))
prob_true, prob_pred = calibration_curve(cal_y, uncalibrated_Pneumonia_probabilities_cal, n_bins=5)
disp = CalibrationDisplay(prob_true, prob_pred, uncalibrated_Pneumonia_probabilities_cal)
disp.plot()
Out[22]:
<sklearn.calibration.CalibrationDisplay at 0x7f55340ffca0>
<Figure size 720x360 with 0 Axes>
Beta Calibration:
We send the uncalibrated Pneumonia probabilties and the true label for each image to a Beta Calibration model that we imported to train the calibration model.
There are several other calibration models such as Platt, Spline, and Isotonic.
For the purpose of this project we only focus on beta calibration.
After fitting the model, we can then use the predict method of the model and use it to convert the uncalibrated probability of the CNN into a calibrated Probability.
Note:
We could do all these inside the training script and deploy the calibration model as part of the endpoint. The calibration dataset could be sent into the training instance in a new channel and everything would work the same way. However, we could not have the visualization and follow the steps. In real production, we would do that as part of the script.In [40]:
from betacal import BetaCalibration
betaCalibration = BetaCalibration()
betaCalibration.fit(uncalibrated_Pneumonia_probabilities_cal, cal_y)
Out[40]:
BetaCalibration()
Visualizing the results
To best compare the results we plot probability diagram for uncalibrated and calibrated probabilties of Pneumonia on the entire test dataset. For this we need to call the endpoint on each image on the test dataset.
Creating the list of images in the test dataset together with their labels:
In [24]:
s3 = boto3.resource('s3')
my_bucket = s3.Bucket(bucket)
Normal_images_test = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/test/NORMAL"):
Normal_images_test.append(object_summary.key)
Pneumonia_images_test = []
for object_summary in my_bucket.objects.filter(Prefix="chestCRXImages/chest_xray/test/PNEUMONIA"):
Pneumonia_images_test.append(object_summary.key)
print(f"Number of Normal Images available for test: {len(Normal_images_test)}")
print(f"Number of PNEUMONIA Images available for test: {len(Pneumonia_images_test)}")
Number of Normal Images available for test: 234 Number of PNEUMONIA Images available for test: 390
In [27]:
test_data = []
test_y = []
for image_name in Normal_images_test:
image_label = 0
test_data.append(image_name)
test_y.append(image_label)
#print('----------------------------')
#print("Image: " + image_name)
#print("Label: " + str(image_label))
for image_name in Pneumonia_images_test:
image_label = 1
test_data.append(image_name)
test_y.append(image_label)
#print('----------------------------')
#print("Image: " + image_name)
#print("Label: " + str(image_label))
print(f"Total size of test dataset is {len(test_data)}")
Total size of test dataset is 624
Calling the endpoint and storing the calculated uncalibrated Pneumonia probabilties. We skip visualization of the images here and will do it at the end.
In [30]:
from sagemaker.serializers import IdentitySerializer
predictor.serializer = IdentitySerializer("image/jpeg")
jpeg_serializer = IdentitySerializer("image/jpeg")
session = boto3.Session()
s3_client = session.client('s3')
s3 = boto3.resource('s3')
from PIL import Image
import numpy as np
bucket_name = bucket
bucket_ = s3.Bucket(bucket_name)
count = 0
uncalibrated_Pneumonia_probabilities_test = []
for file, label in zip(test_data, test_y):
print("--------------------------")
print("Image: " + file)
print("Label: " + str(label))
object = bucket_.Object(file)
response = object.get()
file_stream = response['Body']
img = Image.open(file_stream).convert('RGB')
#display(img)
count += 1
s3_object = s3_client.get_object(Bucket = bucket_name, Key = file)
body = s3_object['Body']
content = body.read()
response = predictor.predict(content, initial_args={'ContentType':'image/jpeg'})
probs = softmax(response[0])
uncalibrated_Pneumonia_probability = probs[1]
print("Uncalibrated Pneumonia Probability: " + str(uncalibrated_Pneumonia_probability))
uncalibrated_Pneumonia_probabilities_test.append(uncalibrated_Pneumonia_probability)
prediction = np.argmax(response[0]) + 1
prediction = 'NORMAL' if np.argmax(response[0]) == 0 else 'PNEUMONIA'
print("Prediction is: " + prediction)
#if count == 5:
# break
-------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1049278-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8147431014939452 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1110860-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.26126209169732845 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-11419-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10093189776727404 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-115218-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4459524603035636 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1160949-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9375761720203815 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1212407-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2524494160190298 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1228182-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21887118664468924 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1283091-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5309149848161783 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1318320-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10242404246439496 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1368583-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6164154598953595 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1430636-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1519676408172442 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1520670-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11014992222202792 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-152130-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21373022107169332 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-159472-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9345202677279567 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1608079-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9762664163387705 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1627110-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7268780915613468 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1698651-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.978289887387899 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-171327-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18345303935393395 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1759114-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.36858404963792324 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1763721-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1443789139696456 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1768815-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.976976935532641 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1771524-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.35856522902480453 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1784004-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.46380132593226253 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1803887-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8315302686103259 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1858497-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16476152207102582 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-186900-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.265372790761402 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1931427-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2930831821553782 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-1944537-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3002217690072368 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2107985-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6332501028370436 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2123652-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9760612924888924 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2162145-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.19797389962793338 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-217318-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.37526594243407335 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2233350-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.13239548434412785 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2256620-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6033092961914704 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2274324-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.12007627097223263 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2280080-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.042962616160498825 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2298727-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9857538652842233 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2403676-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9628586540416588 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2477476-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.0638413965578636 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2514572-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4078806510397887 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2578531-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.03170025980681353 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2704875-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.28001791562954287 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2728578-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3154354878562317 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2771774-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5180563678005933 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2798826-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.05201155358082771 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-283245-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.38854147302362735 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-284601-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9920142834911609 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2850034-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08278823443680658 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2862975-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6761836855274607 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2870844-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.20193618647430034 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2924895-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4135946463842736 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-2959018-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.17744641824295235 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3065672-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.39407156067816007 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3098476-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.17573135080152302 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3117193-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9440120494706454 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3170711-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.43976038648723165 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3189250-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9972815864641986 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3267425-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.365124189924618 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3293997-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.06868912045963804 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3322209-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1897355955822874 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3346259-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9642868165716043 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3395085-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.14283662891002313 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3396581-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21157607009893642 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3397442-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.140156119121892 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3437568-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10321796701179742 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3459196-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09328614996752617 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3464500-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2851896714312553 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3464500-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.22767441342768066 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3464500-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3359567937172916 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3487620-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.38129234282062396 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3518268-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9245753435292091 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3549035-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9898475045372126 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3564811-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6810926003605003 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3624496-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9957235806151649 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3686746-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.843330925753199 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3733696-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07709330074722301 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3782395-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5142490268982204 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3837280-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.048200537789113866 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3921425-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.27429890843935034 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3937418-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.33227683928697466 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3983280-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.908456260955276 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-3985743-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2049376037784618 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4172448-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.22344221180303042 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4190414-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16456558815374336 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4196521-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2892604179551749 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4315572-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8219498789855713 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4340661-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.13481373652815634 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4363211-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10433514657150718 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4511610-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.769134441148862 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4512-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3070449684560613 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4513268-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.26040760394821894 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4534031-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.27698429164167554 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4675782-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6507691556177806 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-472699-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2735235301809359 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4766279-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.862800325821687 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4834475-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5477097262769715 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4852815-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11713456827262474 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4886177-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4226393142728031 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4906464-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10573509360890299 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4911367-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.28669966723794466 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4921478-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7678172197039417 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4955194-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10831415201435228 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5083836-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2553316352811159 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5148117-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2523385386994942 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5175014-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11484184038421995 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5180957-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9127694442101764 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5200363-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.048289226528965 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-520267-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.25556613624595725 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-522391-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5461395451924254 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-528189-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21020871529707272 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5349917-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4918206441663985 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5356114-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9471396834609814 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5388550-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.814555423391448 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5401638-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11198163974982821 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5402107-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07591389411361453 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-541071-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08116303030870284 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5423277-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07466280170041485 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5425295-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.983102686564178 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5459424-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08715806241189292 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5472638-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3260230515834349 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5487025-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9931548643203468 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5498455-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1627692639811311 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5511958-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.453181737091916 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5513974-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1698690182355527 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5538803-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15228517228310998 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5545906-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1851245285854423 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5578360-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5069226668992202 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5601980-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6822533915251328 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5611875-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2049739868495768 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-561684-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.0945540713035013 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5661793-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.12227634052492262 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5661793-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1491160343484463 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5714306-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09463388636228942 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5789014-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15518888554489552 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5889760-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8342420243771473 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5893138-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1756405209223346 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-5997885-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.052791749458375094 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6000527-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09699826717229462 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6085151-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3195373370092793 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6106856-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1243645872463348 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6112608-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.30122605256040264 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6137647-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9087077914189228 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6190523-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5040076112807167 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6195450-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2915127988449494 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6204576-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.10142495887490532 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6267994-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.44415413634625206 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6293151-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.25477114384502664 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6297597-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.22382167937194483 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-630880-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8712555385954821 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6322108-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.20029447046682788 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6332708-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.24607407536991396 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6332708-0002.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.822437452530241 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6332708-0003.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4324461681666481 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6342197-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8237882279833231 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6395296-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7172762632109436 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6410283-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.11556811750971614 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6442221-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9295618513437252 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6497443-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.35632729502169824 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6503977-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.09121222038711452 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6518197-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9912148723805888 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6524124-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9877993935389832 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6544436-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.34710495290322385 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6569259-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21005341097651145 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6650392-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9254488423286533 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6667079-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9100050037698492 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-667173-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9200931205548203 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6703189-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.30870137799061104 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6713743-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.53785497416208 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6874528-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2055223909065608 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6983998-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.15313650355676323 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7031638-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4681656819303341 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-706052-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.25836944356669617 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7082548-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8355077451188432 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-71002-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5977062219044457 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7100645-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8713761626415881 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7165123-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9507210344862049 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7225301-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8927948595203287 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7236364-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4111279961933971 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-724740-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2544369983918707 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7331861-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.22890392726274433 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7398200-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3667735081862521 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7423096-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9843174177807116 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-745902-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3163476046783968 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7483489-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21924165995828726 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7488601-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.29772975239642463 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7519498-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.24345955120513557 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-754920-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4102456482602127 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7590045-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8714675639059734 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7695736-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21611631450232602 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7738170-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.33602031735990134 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7804528-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18346192353529947 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-78265-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.6264462588192173 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-786023-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9771310111321301 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7922918-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18726456138406694 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7950798-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.044592857754042314 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7961936-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.056046146140161554 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8014191-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08146008965691905 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8207090-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.19366237490024027 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8278998-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9918286315951118 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8305321-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08095412725713258 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8305499-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4722195214414475 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8406720-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.956541072630444 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8417023-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9729955636696525 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8418332-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07850138815548977 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8441235-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.0835237102298112 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8448068-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3812534607323749 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8495317-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9519470959070799 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-857568-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3756656779358182 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8586608-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2000505918428751 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8661138-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.171821953929319 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8698006-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4143422915852774 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-870639-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.19106033707315775 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8750803-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7517363691228751 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8836138-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7431183291168711 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8876914-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.1492213070095748 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8907762-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5122784272886866 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-8944072-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.20706083577045395 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9054075-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.9988483665188695 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9077486-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.4866504365759639 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9092354-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.26409409889982344 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9104493-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.08976426674599758 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9145111-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.2770667994754604 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9194822-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.21487035554687783 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9217241-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3136003501731313 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9260113-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.06763408000562773 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9275512-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.5948746625600907 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9334007-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3127383690140007 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9343440-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07740189207238869 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9441169-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.3337086115746784 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9708691-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.16600538727010342 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9717513-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.18193420497797 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-976006-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.7088676103370273 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9799657-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.8186478499830165 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-9896495-0001.jpeg Label: 0 Uncalibrated Pneumonia Probability: 0.07674111117809021 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1135262-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9724986771549766 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1135262-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9817690262076074 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1135262-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9968367721523199 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1135262-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9373627765046999 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1220485-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994570371002236 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992081429207745 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999695349815599 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9955128008658252 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999237508582602 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9989653809809362 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1351146-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9325551813947633 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1514320-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9877758066120408 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1514320-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5960501873943282 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1514320-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8170560017523872 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1602272-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981454640403801 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1602272-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996396130677092 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1602272-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9934236077992445 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1602272-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.618460528270965 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1714895-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999065975971743 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1714895-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997647697736528 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1714895-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.30508648826217466 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9796183429668995 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9770385426154212 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9704006497566654 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.998474895602483 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9856908365206151 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-1768914-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.996796411327245 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.4935265150066973 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999628177790041 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996153257102737 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999348297455227 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985664951672554 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2034017-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9941662553139972 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2131702-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9943448584568877 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2131702-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995326055787108 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2131702-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9944477219202033 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-227418-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9968244218872427 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-227418-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.993414129116223 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-227418-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9935215283863145 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-227418-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996999251210773 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2429208-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.992679705461366 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2429208-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9967211882477691 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2429208-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5119870735650647 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2772694-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9953367744339288 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-2772694-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996266479464135 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3060399-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9940544008372936 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3060399-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.997780346801358 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3060399-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9934223974157452 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3060399-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9784663090655025 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3105669-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9979984777381361 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3105669-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9762338208966395 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3105669-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996502160283052 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3105669-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9909759069775311 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3105669-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9334907081102809 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3127602-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.958726172633623 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3127602-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9782948416856886 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3436159-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9939712913765212 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3436159-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9967612616871747 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3436159-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.409036502610057 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3865791-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9946370524061198 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3865791-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9837160706758913 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3865791-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9914654674598946 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3865791-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.998869428955989 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3865791-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9978200697086761 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3961172-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9807073857679895 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3961172-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981009594059287 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3961172-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9779009927925785 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-3961172-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5809428544559254 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4006043-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9941812032278443 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4006043-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9891733416416445 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4006043-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9987614088562389 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4059589-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9988721775601832 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4059589-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9965759303400042 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4059589-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9965759303400042 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4059589-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9420609255211916 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-40699-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9878898417522148 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-40699-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5051966161257193 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4070745-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9971186258786435 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4130801-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.852838429202703 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4130801-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996436213572637 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4130801-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8761658047523171 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4161430-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997406129720136 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4161430-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999939346860884 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4161430-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993989241929812 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4167818-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995966911068505 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4167818-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9504655474477173 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4167818-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9991658551488299 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9180589033713997 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998017821542353 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9812480431149659 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9935353234721589 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8639805725013259 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9982403609739121 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994333376713952 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0008.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999085842401563 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4269599-0009.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5035775661874726 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4322734-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999700234820915 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4322734-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995838993903973 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4345731-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998621970944168 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4345731-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9958181035660585 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4345731-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9970808683452671 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4345731-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5035775661874726 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-4376803-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9470008849662949 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5141192-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9857602019534273 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5141192-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999939542275422 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5141192-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.27209695291704833 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5155977-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993188572959535 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5155977-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9947960520463174 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5155977-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9942926331368303 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5155977-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985139073220717 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5155977-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999750582799447 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-518323-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999568277764013 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-518323-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9802623110211265 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-518323-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9566973054544398 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5240350-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9982551489735588 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5240350-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985591325969863 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5240350-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998947155535532 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5240350-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8250708137547713 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5421131-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996232795159742 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5421131-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9976145240847647 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5451280-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999866863822608 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5451280-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7199699292502154 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-545831-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992555543786554 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5489623-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9928992563530139 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5489623-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996844051689061 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5489623-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7416126762596313 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999334003873722 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995418121394765 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9945140499799856 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9973957410939032 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9987907533371809 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9911801632189188 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994965219107351 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5660566-0008.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7660620023321344 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5754167-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.989987515353521 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5754167-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7845690453745594 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-5757955-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.17106837590867321 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6105615-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9983260167198912 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6105615-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.998882289564755 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6168941-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997594048465168 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6168941-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9890755850251928 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6402155-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997485341703131 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6402155-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7754125349997081 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6402155-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994646383161038 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6402155-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.31044148592666715 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6523466-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9986229960889005 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6523466-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997922380807509 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6806075-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8314177741890567 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6806075-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9982370334630376 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6806075-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995335897280567 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6806075-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5152824866923542 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6806075-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6961168391536314 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6833685-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9970267794460086 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6833685-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9904426907338169 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6833685-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.3871323725054439 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5035775661874726 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9590947101802051 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9950441783458212 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998781880028338 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9954588121294761 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9949295964916074 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6950003-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9981460884256486 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-6987875-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.933471888074465 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7051071-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999869739274508 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7051071-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999951311471727 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7098043-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992844439000129 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7136666-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985213591401644 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7257218-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5035775661874726 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7257218-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9979559322224988 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7257218-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9976073631973446 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7257218-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.997968500121552 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7257218-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5126582152714428 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7346761-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.999825925861841 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7720431-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9931785251035173 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7720431-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9950763190585836 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7720431-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.996875957155867 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-7720431-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.4330470024574012 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8046863-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9961234657490103 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8046863-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999596713351706 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8190872-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8919667560373373 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8190872-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9900797708487638 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8190872-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992470189413512 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8190872-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993263205856849 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8190872-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9859486968888248 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-821920-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995241032738775 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-821920-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.4831951845738851 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8285922-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9950437115693201 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8285922-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9909351698610604 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8285922-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9929113538343576 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8285922-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9975013303668894 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8285922-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9975013303668894 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-840611-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.997461766835255 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-840611-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994898621999505 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-840611-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999704443164803 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-840611-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996244803362176 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-840611-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7221940202056395 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-846757-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9968734702696905 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-846757-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998838732155345 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8563288-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994610826208801 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8837254-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9988789888022563 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8846908-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992957262677246 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8846908-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9910289222697741 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8846908-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5426667157939747 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8983865-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998863855557135 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8983865-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997680346342486 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8983865-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998018357278715 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8983865-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.992830198670639 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-8983865-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6001416416090852 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9050379-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.989667433156452 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9060950-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9986770981967484 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9060950-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9761852435149014 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9156003-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995471196045291 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9156003-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999194799066883 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9156003-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997001646240349 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9156003-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993301639024739 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9156003-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.30594955061415646 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9182718-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9961897716645169 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.4863539864966656 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9929637201302026 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999904053064375 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9832877444642535 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999877266646012 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9242636-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.452059350854646 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-932292-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9950114632117358 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-932292-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9997803250612494 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-932292-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9998102845693502 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9364007-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999282922149024 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9364007-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9990507272686048 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9364007-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.29195800483215084 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-949117-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994720587038202 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9503383-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995323785241751 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9673743-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995540631368922 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9673743-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996781916725513 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9673743-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9922975660546463 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9764136-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9778325637665531 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9857270-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9999745721452173 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9857270-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9975287672190583 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9857270-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7996756496842428 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9857270-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.31164906749317395 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9913238-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9987030775384518 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9913238-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.983520496586147 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9913238-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9657993350335129 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1056329-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9029971593569295 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1142234-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6956003937813624 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1207991-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9747354610057298 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-131422-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.952624611665074 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9995527306226653 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994907511538994 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8273257775620878 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0004.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9952481884122429 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0005.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9992433472503917 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0006.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9991257801887512 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1352878-0007.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9983592867000359 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1423083-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9988203437487846 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1463523-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9881649236744016 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1517175-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9550001986218031 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1540910-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9902016364793278 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1540910-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9896148348947803 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-1795675-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9963564165873553 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2040583-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9971853984294947 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2040583-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993047448665303 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2061674-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9929741951085714 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2102459-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9824037254725297 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2344316-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9890229080647098 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2352342-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9973798621802573 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2352342-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.998777586491376 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2470676-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9958234219909085 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-258000-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9082728730009018 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-262135-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9417800974374373 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-280992-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9639794011101412 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2831144-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9337265099106995 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-2914571-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9977081251383203 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3008952-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9944785093193932 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-30184-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5219862959553183 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3028666-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9898283452707017 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3039338-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9930898493560543 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3119878-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.590722016784309 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3174339-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8646593040825803 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3225323-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9938683316788399 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3239718-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9758734372045894 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3239718-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9984800915538793 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3426682-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9945425205068542 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3439481-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9848041665093916 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3522052-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9940414511807156 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3565661-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.84447930754305 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3635859-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9433563085541218 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3698459-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9010886521380933 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3746644-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9800151585316698 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3768496-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9691475520390852 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3814177-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9449411249055966 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3828598-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9947416737041495 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3828598-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9979127871692814 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3835085-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9816859989583031 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3835085-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9975044202940893 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3838349-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9983272733289333 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3906469-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.5498699576400361 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-3915569-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.98588339905186 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4009702-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9888743718900206 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4033321-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.99972486198085 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4190128-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.994341267019794 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4275213-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7345342684566623 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4276353-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.997012291053177 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-432590-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9985923878722557 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4355102-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9936102049044002 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4360206-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9966706727567515 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4405644-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9559352159088326 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-447130-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996429949917757 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4533169-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8162120644605894 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4655012-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9680358471705914 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4664910-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996153520210227 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4810299-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.878546206416458 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4810299-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9908273876241344 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-48197-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9993920036022208 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4840553-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9928442916766663 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4852350-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9749879833645084 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-490019-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994406531845116 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-490019-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994406531845116 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-4911614-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9515828724430084 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5069337-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9695521009389901 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5180281-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9900469920421163 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5271295-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9072062868044121 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5366974-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9692456861047722 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5391810-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9994918196334167 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5508710-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9570982952629105 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5660566-0009.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9940367425902835 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5708148-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9932923430091466 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5708148-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9877919320400034 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5741681-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9715180401187615 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5813112-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9413762522417102 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5864551-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9959407569880987 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5880719-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9819035220350891 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-5927483-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9953312502773712 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6076183-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8482478023017853 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6076183-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6756302797403237 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6079770-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9143229718370065 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6138035-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9978511386493079 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6207158-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9955222973156591 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6249454-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9974992318185514 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6249454-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9903159326174356 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6249454-0003.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9742858864997389 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6682185-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9500706121932766 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6733553-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9964288767729759 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6812602-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9904787780262664 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6828836-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7357545911906386 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6831597-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9824296636184396 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6840343-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9551737023848484 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6844248-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7363233991528467 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-6844248-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9612959238959358 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7014527-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9699627509080881 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7014527-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9951871399616402 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7318158-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8877536219090787 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7337079-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.988605294389708 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7385225-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.7063010476424256 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7441806-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9955907188077975 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7441806-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9885774511850087 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7477247-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.3922046950491694 Prediction is: NORMAL -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7589381-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9784764717258372 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7638941-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8927409078288708 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7688430-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9370492792339112 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7788460-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9990627436484621 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7822291-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.993726482200653 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7909775-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9827694294382227 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-7979-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9989240944561871 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8106836-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9891362492737271 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-820420-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9033595934109355 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8314555-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.6810705650894825 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8377377-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9958713788391991 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8385372-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9979548055317571 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-842024-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9663917290707738 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8427429-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9968257604659837 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8435135-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9028324436932745 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8507816-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9964527585380804 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8544560-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9979150761965347 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8574414-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9949068935014239 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-874167-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9901389540607524 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-8903181-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9827040711939335 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9018258-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9940078594739867 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9039335-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9703134409306517 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-904779-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9763711505723708 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9240664-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.8172228608961446 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9325276-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.987209766881511 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9342284-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9823872258680323 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9476607-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9990661511590725 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9564688-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9996939252589763 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9564688-0002.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.994890511999061 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9584831-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9917859201522394 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9671740-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9905110437099386 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9783315-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9951915721314664 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9890836-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.9983520627700052 Prediction is: PNEUMONIA -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/VIRUS-9968655-0001.jpeg Label: 1 Uncalibrated Pneumonia Probability: 0.894028663904279 Prediction is: PNEUMONIA
Visualizing uncalibrated probability diagram
Using ml_insight library:
In [34]:
import ml_insights as mli
uncal_prob_test = np.array(uncalibrated_Pneumonia_probabilities_test)
y_test = np.array(test_y)
plt.figure(figsize=(10,5))
mli.plot_reliability_diagram(y_test, uncal_prob_test, show_histogram=True);
Just a side note:
ml_insights library sometime has issues when importing on AWS. We could do the same plots using scikit-learn. Example shown below:
In [49]:
plt.figure(figsize=(10,5))
plt.hist(x=uncalibrated_Pneumonia_probabilities_test, bins=10, color='#0504aa', alpha=0.5, rwidth=0.75);
plt.figure(figsize=(10,5))
prob_true, prob_pred = calibration_curve(test_y, uncalibrated_Pneumonia_probabilities_test, n_bins=10)
disp = CalibrationDisplay(prob_true, prob_pred, uncalibrated_Pneumonia_probabilities_test);
disp.plot()
Out[49]:
<sklearn.calibration.CalibrationDisplay at 0x7f5533a79f40>
<Figure size 720x360 with 0 Axes>
In [51]:
p1 = np.array(prob_true)
p2 = np.array(prob_pred)
error = p1 - p2
error_square = np.square(error)
error_sum_square = np.sum(error_square)
print(f"Uncalibrated probability Sum of Squares: {error_sum_square}")
Uncalibrated probability Sum of Squares: 0.29199751521036654
We note from the result that the uncalibrated probabilities are highly over-estimated as we expected from visualizing the same on the smaller calibration dataset.
From histogram, we note that we dont have many points in probability regions around 0.6 to 0.8 and this results in noisy points in the calibration diagram. This is an issue with this dataset as even looking at the entire dataset, there are not enough points. However, this does not much affect overal performance and we can focus on regions with more datapoints.
We also defined a metric as Sum of Squared Error that illustrated the difference between predicted and actual probabilities.
Visualizing uncalibrated probability diagram
Now, we send the uncalibrated probabilities to the calibration model to get the calibrated probabilties and then visualize the output.
Using scikit-learn:
In [52]:
calibrated_Pneumonia_probabilities_test = betaCalibration.predict(uncalibrated_Pneumonia_probabilities_test)
plt.figure(figsize=(10,5))
prob_true, prob_pred = calibration_curve(test_y, calibrated_Pneumonia_probabilities_test, n_bins=10)
disp = CalibrationDisplay(prob_true, prob_pred, calibrated_Pneumonia_probabilities_test)
disp.plot()
Out[52]:
<sklearn.calibration.CalibrationDisplay at 0x7f553037b520>
<Figure size 720x360 with 0 Axes>
Using ml_insight library:
In [53]:
cal_prob_test = np.array(calibrated_Pneumonia_probabilities_test)
mli.plot_reliability_diagram(y_test, cal_prob_test);
In [54]:
p1 = np.array(prob_true)
p2 = np.array(prob_pred)
error = p1 - p2
error_square = np.square(error)
error_sum_square = np.sum(error_square)
print(f"Calibrated probability Sum of Squares: {error_sum_square}")
Calibrated probability Sum of Squares: 0.22386608494114665
We can notice much improvement in the diagram. The points from the model are generally closer to the actual probability.
Our Sum of Squared Error metric is also improved. Although the difference seem small but it is a significant improvement.
Applying calibration to the endpoint in production
As noted earlier, we could do the probability calibration inside the script and deploy that together with the CNN model to the endpoint. But we did this in the notebook instance for illustration.
This could still be a good choice becuase the calibration model can later on be applied to the logits received from the endpoint. Another option is that a Lambda function interfacing with the endpoint can apply the calibration to the endpoint result and return the calibrated probabilities to the client.
For illustration, we randomly select 5 X-ray images from the test dataset, call the endpoint, apply the calibration to the response from the model, and report both calibrated and uncalibrated probabilities. For visualization, the image used for this inference is also shown.
In [57]:
from sagemaker.serializers import IdentitySerializer
predictor.serializer = IdentitySerializer("image/jpeg")
jpeg_serializer = IdentitySerializer("image/jpeg")
session = boto3.Session()
s3_client = session.client('s3')
s3 = boto3.resource('s3')
test_len = len(test_data)
sample_index = random.sample(range(test_len), 5)
sample_images = [test_data[index] for index in sample_index]
sample_labels = [test_y[index] for index in sample_index]
from PIL import Image
import numpy as np
bucket_name = bucket
bucket_ = s3.Bucket(bucket_name)
uncalibrated_Pneumonia_probabilities_test = []
for file, label in zip(sample_images, sample_labels):
print("--------------------------")
print("Image: " + file)
print("Label: " + str(label))
object = bucket_.Object(file)
response = object.get()
file_stream = response['Body']
img = Image.open(file_stream).convert('RGB')
display(img)
count += 1
s3_object = s3_client.get_object(Bucket = bucket_name, Key = file)
body = s3_object['Body']
content = body.read()
response = predictor.predict(content, initial_args={'ContentType':'image/jpeg'})
probs = softmax(response[0])
uncalibrated_Pneumonia_probability = probs[1]
print("Uncalibrated Pneumonia Probability: " + str(uncalibrated_Pneumonia_probability))
calibrated_Pneumonia_probability = betaCalibration.predict([uncalibrated_Pneumonia_probability])
print("Calibrated Pneumonia Probability: " + str(calibrated_Pneumonia_probability[0]))
prediction = np.argmax(response[0]) + 1
prediction = 'NORMAL' if np.argmax(response[0]) == 0 else 'PNEUMONIA'
-------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-6395296-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.7172762632109436 Calibrated Pneumonia Probability: 0.727611282798618 -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-7165123-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.9507210344862049 Calibrated Pneumonia Probability: 0.9396063458887806 -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-630880-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.8712555385954821 Calibrated Pneumonia Probability: 0.8828038394175102 -------------------------- Image: chestCRXImages/chest_xray/test/NORMAL/NORMAL-4315572-0001.jpeg Label: 0
Uncalibrated Pneumonia Probability: 0.8219498789855713 Calibrated Pneumonia Probability: 0.8416779295391046 -------------------------- Image: chestCRXImages/chest_xray/test/PNEUMONIA/BACTERIA-9673743-0001.jpeg Label: 1
Uncalibrated Pneumonia Probability: 0.9995540631368922 Calibrated Pneumonia Probability: 0.9928922463476831
In [ ]:
References:
- Medical imaging in personalized medicine: a white paper of the research committee of the European Society of Radiology (ESR). Insights Imaging 6, 141–155 (2015). https://doi.org/10.1007/s13244-015-0394-0
- E. Sogancioglu et. al., Deep Learning for Chest X-ray Analysis: A Survey, Medical Image Analysis, arXiv:2103.08700
- Raoof, S., Feigin, D., Sung, A., Raoof, S., Irugulpati, L., Rosenow, E.C., 2012. Interpretation of plain chest roentgenogram. Chest 141, 545–558. doi:10. 1378/chest.10-1302 Balabanova, Y., Coker, R., Fedorin, I., Zakharova, S., Plavinskij, S., Krukov, N., Atun, R., Drobniewski, F., 2005. Variability in interpretation of chest radiographs among russian clinicians and implications for screening programmes: observational study. BMJ 331, 379–382. doi:10.1136/bmj. 331.7513.379
- Kermany DS, et. al., Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell. 2018 Feb 22;172(5):1122-1131.e9. doi: 10.1016/j.cell.2018.02.010. PMID: 29474911.
In [ ]:
In [ ]: